{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "tags": [
     "hide-input"
    ]
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "import os\n",
    "if not any(path.endswith('textbook') for path in sys.path):\n",
    "    sys.path.append(os.path.abspath('../../..'))\n",
    "from textbook_utils import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lLVVSGne3AIY"
   },
   "source": [
    "# Exercises"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Use `re.search` to find whether a string contains a phone number. The pattern that you write should detect a phone number in the following strings.  \n",
    "\"Call me at 382-384-3840.\"  \n",
    "\"my number is (510) 849-3519. Call me!\"  \n",
    "And not find a match in the following strings.  \n",
    "\"my number is 510-849-35192\"  \n",
    "\"here's my number: 510-849.3519\"  \n",
    "Consider making your own tests as well. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Use `re.sub` to alter the string below so that the dates have a common format that uses a dash for the day, month, and year separator.  \n",
    "'03/12/2018, 03.13.18, 03/14/2018, 03:15:2018'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Use `re.split` to separate the chapter name from the page number in the following table of contents for a book. \n",
    "```\n",
    "toc = '''\n",
    "PLAYING PILGRIMS ..............  3\n",
    "A MERRY CHRISTMAS ............. 13\n",
    "THE LAURENCE BOY .............. 31\n",
    "BURDENS ....................... 55\n",
    "BEING NEIGHBORLY .............. 76\n",
    "'''\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Consider the first five sentences of the novel \"Little Women\" below. Extract the spoken dialog from each sentence.  \n",
    "```\n",
    "text = '''\"Christmas won't be Christmas without any presents,\" grumbled Jo, lying on the rug.\n",
    "\"It's so dreadful to be poor!\" sighed Meg, looking down at her old dress.\n",
    "\"I don't think it's fair for some girls to have plenty of pretty things, and other girls nothing at all,\" added little Amy, with an injured sniff.\n",
    "\"We've got Father and Mother, and each other,\" said Beth contentedly from her corner.\n",
    "The four young faces on which the firelight shone brightened at the cheerful words, but darkened again as Jo said sadly, \"We haven't got Father, and shall not have him for a long time.\"'''\n",
    "```   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Return to the Web log example and use regular expressions with grouping to extend the pattern we used to extract the day, month, and year, to also extract the hour, minute, and second from the log. How would you also extract the time zone?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Return to violation descriptions for the restaurant inspection (see {numref}`Chapter %s <ch:wrangling>` and derive some of your own features for exploring the relationship between the kinds of violations and the inspection score."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- In text analysis, stop words are often removed because they are commonly used and thought to contain little information. However, the use of these words can be insightful in determining the authorship of documents. As an example, the Federalist papers were published anonymously in 1787-1788 by Alexander Hamilton, John Jay, and James Madison. All together 77 papers were published. It is generally agreed that Jay wrote five of the essays (2, 3, 4, 5, and 64), Hamilton wrote 43 papers , Madison of 14 (10, 14, and 37-48), and three were co-authored by Hamilton and Madison (18-20). However, the authorship of 12 papers (49-58, 62, and 63) is in dispute between Hamilton and Madison. The Federalist papers are available on line in Project Gutenberg at https://www.gutenberg.org/ebooks/18. Mosteller and Wallace [^FP] have found that the words: *to, this, there, on, of, by, a, also* are excellent at discriminating between Hamilton and Madison authored papers. Carry out your own text analysis to confirm these findings and predict the author of the 12 papers with disputed authorships.  \n",
    "\n",
    "[^FP]: https://www.jstor.org/stable/2283270"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "DAP-lNNw3AKV"
   },
   "source": [
    "- Rotten Tomatoes [^RT] is a recommender system for movies and TV shows. It includes a Tomatometer where negative reviews are considered rotten and positive ones fresh. Reviews include scores (rated 1 to 5), comments, and a rotten/fresh designation. Data for 17,000 movies are available on Kaggle [^RTDB] and another 2,000 one-sentence reviews from Rotten Tomatoes are also available [^CornellMovies] for analysis. Explore the relationship between the comments and scores and designation. Try detecting the sentiment of a written review. Socher et al. [^Treebank] found that the bag of words approach to analyzing text for sentiments does not work well on short text, such as movie reviews, and recommend considering word order. \n",
    "\n",
    "[^RT]: https://www.rottentomatoes.com/\n",
    "[^RTDB]: https://www.kaggle.com/datasets/stefanoleone992/rotten-tomatoes-movies-and-critic-reviews-dataset\n",
    "[^Treebank]: https://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf\n",
    "[^CornellMovies]: https://www.cs.cornell.edu/people/pabo/movie-review-data/review_polarity.tar.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "RTJu-FQY3AIp",
    "TrPjIHHD8rOC"
   ],
   "default_view": {},
   "name": "working_with_text.ipynb",
   "provenance": [],
   "version": "0.3.2",
   "views": {}
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  },
  "toc": {
   "nav_menu": {},
   "number_sections": false,
   "sideBar": false,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": true,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
